Field of the Invention
This invention relates generally to the field of computer processors. More particularly, the invention relates to a virtual reality apparatus and method for prioritizing pixel shader operations based on early depth testing, performing alternate eye rendering, and/or augmented timewarp operations.
Description of the Related Art
For accelerated rendering, it is common to perform a depth prepass, sometimes referred to as “Z-prepass.” The reason for this is that the GPU or graphics processor should ideally perform pixel shading only for visible surfaces. When a scene is rendered without a Z-prepass, a triangle that is far away may be rendered first and hence pixel shading will be performed, and later a closer triangle may overwrite that far-away triangle with the pixel shading of the closer triangle. Hence, the work done on the far-away triangle was done in vain since it did not contribute to the image. Instead, it is common to render the scene twice using a Z-prepass as a first pass. In the first pass, the scene is rendered but only depth is written to the depth buffer and no pixel shading is performed nor is anything written to the color buffer. As a result, when the first pass has ended, the depth buffer contains the depth of the closest surface at each pixel. The second pass renders all the triangles with pixel shading on, depth writes turned off and the depth test as EQUAL, i.e., color is only written if the fragment has the same depth as the depth in the depth buffer. This means that all fragments of rendered triangles that are farther away than the depths in the depth buffer will NOT perform any pixel shading, i.e., pixel shading will only be performed on the closest surface in each pixel, resulting in more efficient pixel shading. In addition, all graphics architectures have some form of hierarchical depth buffer with culling, such as the HiZ buffer, and the first pass will “prime” the HiZ-buffer (typically a Zmin and Zmax value per 8×8 pixels) and hence, occlusion culling can be done efficiently in the second pass using the HiZ buffer. In the example above, it is assumed that all geometry/triangles are opaque.
In addition, current virtual reality (VR) systems render separate image streams for the user's left and right eyes, thereby consuming roughly twice the graphics processing resources and/or requiring twice the amount of time as would be required to process a single image stream.
Time warping is a technique used to improve performance in current virtual reality (VR) systems. According to this technique, each image frame is rendered in accordance with the current orientation of the user's head and/or eyes (i.e., as read from an eye tracking device and/or other sensors on the head mounted display (HMD) to detect the motion of the user's head). Just before displaying the next image frame, the sensor data is captured again and us used to transform the scene to fit the most recent sensor data (i.e., “warping” the current image frame). By taking advantage of the depth maps (i.e., Z Buffers) which have already been generated, time warping can move objects in 3D space with relatively low computational requirements.